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Abstract 

Background: Resource-limited countries increasingly depend on quality indicators to improve outcomes within HIV 
treatment programs, but indicators of program performance suitable for use at the local program level remain 
underdeveloped. 

Methods: Using the existing literature as a guide, we applied standard quality improvement (Ql) concepts to the 
continuum of HIV care from HIV diagnosis, to enrollment and retention in care, and highlighted critical service 
delivery process steps to identify opportunities for performance indicator development. We then identified existing 
indicators to measure program performance, citing examples used by pivotal donor agencies, and assessed their 
feasibility for use in surveying local program performance. Clinical delivery steps without existing performance 
measures were identified as opportunities for measure development. Using National Quality Forum (NQF) criteria as 
a guide, we developed measurement concepts suitable for use at the local program level that address existing 
gaps in program performance assessment. 

Results: This analysis of the HIV continuum of care identified seven critical process steps providing numerous 
opportunities for performance measurement. Analysis of care delivery process steps and the application of NQF 
criteria identified 24 new measure concepts that are potentially useful for improving operational performance in 
HIV care at the local level. 

Conclusion: An evidence-based set of program-level quality indicators is critical for the improvement of HIV care in 
resource-limited settings. These performance indicators should be utilized as treatment programs continue to grow. 
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Background 

HIV care in low and middle income countries 

Since first reported in 1981, HIV/ AIDS has claimed the 
lives of nearly 27 million people worldwide [1,2]. The 
United Nations Development Program named HIV 
responsible for "the single greatest reversal in human 
development" in modern history [3]. In response, the 
international community has committed to reversing the 
toll of this pandemic through financial support, political 
advocacy, and civic engagement [4]. With these efforts, 
6.7 million people living with HIV in low-and middle-in- 
come countries now have access to lifesaving treatment, 
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representing a 16-fold increase in 7 years [5]. The intro- 
duction of antiretroviral therapy (ART) has been esti- 
mated to have averted 2.5 million deaths in those 
settings [5]. 

This progress has been inconsistent across communi- 
ties and countries. HIV remains under-diagnosed, many 
patients present late to care or not at all, some do not 
receive therapy despite clinical eligibility, and others do 
not remain in care over time [1,6-12]. Addressing these 
challenges will require provision of new services in 
addition to improvement of the quality of existing ser- 
vices. The latter suggests an opportunity to apply the 
tools of quality improvement (QI) championed in well- 
resourced nations [4,13]. 

Several major global HIV donor and advocacy agencies 
have worked to consolidate performance indicators for 
HIV programs [1]. Increasing attention has been paid to 
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the quality and impact of HIV services, and to greater 
alignment with host-country goals. However, several 
weaknesses remain. The strategic framework for moni- 
toring and evaluation utilized by donor and advocacy 
agencies may not always be consistent with the improve- 
ment model where the focus of measurement is on 
process and outcome measures, and change is implemen- 
ted at the smallest replicable unit - the microsystem of 
healthcare delivery. National and global indicators pro- 
vide a broad picture of the scope, size, and impact of 
HIV services, and are important for strategic planning 
and resource allocation. Therefore, national and global 
indicators should not be replaced by program-level indi- 
cators. However, program-level indicators remain incom- 
plete, and a carefully selected set of measures collected at 
6 to 12-month intervals provide important data that can 
be disseminated to staff to guide QI efforts [1]. 

Among the available quality indicators for HIV, several 
critical points including linkage to care, ART eligibility, 
and ART preparations as well as important patient 
reported and clinical outcomes are either not repre- 
sented, or are poorly represented. In addition, current 
indicator and reporting requirements are often decentra- 
lized and dependent on donor agencies [14]. In a study 
of data quality from the international ART-LINC group, 
67% of sites received funding from two sources, and 24% 
from three [14]. In the same study, only 61% of pro- 
grams used the same software for data collection and 
management, creating a gap between measurement, clin- 
ical care and improvement, while increasing the cost 
and complexity of monitoring and evaluation [14]. This 
study suggests that competing priorities from multiple 
funding sources create barriers to data collection on 
additional indicators that may be crucial to local im- 
provement and underscores the importance of identifying 
a set of measures that will meet reporting, stakeholder, 
and local quality improvement needs. 

The aim of this manuscript is to help care delivery 
programs better utilize quality improvement methods by 
providing them with a better ability to assess their own 
performance. More specifically, we propose a framework 
for considering performance measures based on the con- 
tinuum of HIV care. We then identify where gaps exist 
between available measures and those measure concepts 
identified by our framework. 

To accomplish this aim, we first provide an overview 
of quality improvement (QI) science in healthcare, high- 
lighting important QI concepts. We then summarize the 
global HIV-specific QI measurement efforts in resource- 
limited settings utilized by three pivotal advocacy and 
donor agencies with broad international reach (UNAIDS, 
the Global Fund to fight AIDS, Tuberculosis, and Malaria, 
and the President's Emergency Plan for AIDS Relief 
[PEPFAR]). We describe performance along the HIV 



care continuum, identifying potential foci for quality im- 
provement, and gaps in current measurement efforts. 
Finally, we propose a set of quality indicators for HIV 
service delivery programs to address current measurement 
gaps along the HIV care continuum. 

Methods 

To summarize HIV QI concepts and identify existing 
performance assessment measures, we conducted a li- 
terature review using Medline and EMBASE databases in 
addition to published reports from UNAIDS, the Global 
Fund, and PEPFAR. Using available literature, we devel- 
oped an HIV care continuum framework, paying close at- 
tention to key process steps and outcomes susceptible to 
measurement as called for in process improvement 
theory. Having identified the key process steps required 
to deliver HIV services, we again searched the literature 
for studies addressing performance in each of the identi- 
fied process steps. We also assessed the extent to which 
existing performance measures addressed process issues 
identified at each of the process steps. We used the 
literature to propose additional measures and assessed 
the extent to which each met National Quality Forum 
(NQF) recommendations for measure importance, 
usability and feasibility [15]. Assessment of measurement 
properties (i.e. validity, reliability, etc.) was beyond the 
scope of this work. 

Results 

The science of quality improvement in healthcare 

Borrowing from industrial quality control methods devel- 
oped in the 1950s and 1960s, healthcare in resource-rich 
countries has seen rapid growth in the understanding 
and development of methods that improve quality [16]. 
Quality improvement relies both on a philosophy, the 
pursuit of continuous performance improvement, and a 
diverse armamentarium of methods. These methods 
include operations research, assessment of healthcare 
delivery processes, participative management, and setting 
of performance benchmarks. They also include determi- 
ning best practices, often codified in practice guidelines, 
and measuring adherence to those best practices [17]. 

Central to achieving high quality of care is the defin- 
ition, observation and measurement of key system indi- 
cators - the fundamental metrics in quality improvement 
measurement. Many healthcare QI experts support the 
Donabedian model, which suggests that indicators 
should assess the structures (characteristics of physical 
deployment of resources such as physicians, nurses, 
buildings and supplies), processes (what health profes- 
sionals do with people), and outcomes of care (what hap- 
pens to people, especially with regard to their health) 
[18]. These attributes of a system of care are not of equal 
value. Outcomes matter most, but are also the most 
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challenging to measure. Process measures are a better 
indicator of healthcare quality than structural measures 
because they allow assessment of timeliness and appro- 
priateness of care [19]. While individual indicators may 
reflect specific measures of importance, they may not in 
isolation lead to an understanding of the broader land- 
scape and affect change that leads to improvement. To 
address this, the "balanced scorecard" approach was 
designed to merge a set of measures (including struc- 
tures, processes and outcomes) across several domains 
(e.g., timeliness, cost-effectiveness, appropriateness) in a 
format that would enable managers to translate the orga- 
nization's mission into a specific subset of measurable 
objectives [20]. 

The development and selection of indicators is linked 
to data collection, implementation of targeted activities, 
and evaluation of their impact on health outcomes 
[16,21]. Once identified, quality indicators should be 
evaluated for their importance, scientific acceptability, 
usability, and feasibility [22]. These measures can then 
be used at the program level (the point of service deliv- 
ery) to assess healthcare quality and identify areas for 
improvement. This process is most effectively executed 
at the level of the microsystem - a small group of people 
who work together to provide care to patients; who best 
understand the challenges to improvement; and are 
responsible for implementing the changes that drive im- 
provement [23]. For HIV treatment programs, such a 
team may consist of clinicians (physicians, nurses, phar- 
macists, aides, etc.), counselors, home-based care/out- 
reach workers, and clerical/data entry staff. 

HIV-specific Ql measurement in resource-limited countries 

The Joint United Nations Program on HIV/AIDS 
(UNAIDS), along with major funding bodies, has pro- 
vided guidance and leadership to improve the capacity 
and quality of HIV care globally. UN member states 
pledged to regularly report progress on their commit- 
ment to reversing the spread of HIV/ AIDS; in 2003, the 
UNAIDS Secretariat developed a set of core indicators to 
monitor that progress [1,13]. The largest multilateral 
donor agency for the HIV response, the Global Fund to 
fight AIDS, Tuberculosis and Malaria (GFATM) was 
established to increase financing for these infections in 
2002 [24]. The US President's Emergency Plan for AIDS 
Relief (PEPFAR), the largest bilateral funding organiza- 
tion for the global HIV/ AIDS pandemic, was launched by 
the US Congress in 2003 [25-27]. 

UNAIDS, the Global Fund, and PEPFAR have adopted 
an overarching framework for the monitoring and evalu- 
ation of national HIV/ AIDS programs in host countries. 
This framework relies on a logic model that measures 
inputs, activities, outputs, outcomes, and impacts, which 
provide information on what goes into a program and 



the results achieved [28]. The measures are indicators 
collected at the program, provincial, and local levels, and 
are aggregated to reflect the progress of national HIV/ 
AIDS programs in meeting country, donor, and advocacy 
goals. They are multifunctional and provide guidance on 
the strategic planning, coordination and implementation 
of programmatic efforts for the global HIV/AIDS 
response, accountability to donor agencies, and assess- 
ments of program effectiveness [28-31]. 

Despite attempts to align monitoring and evaluation 
efforts, the UN and others, recognizing a lack of uni- 
formity in key system measures, published a set of 25 
core indicators in 2007 on which all countries were to 
report (Tables 1) [1]. GFATM-supported programs report 
on a minimum set of 40 national indicators; 25 of these 
are from the UN General Assembly requirements, and 
an additional 15 indicators span antiretroviral therapy 
treatment programs, development of HIV workplace 
programs, and reduction of stigma (Tables 1) [28]. While 
some indicators were appropriate for program-level 
monitoring, many were not, and the Global Fund recog- 
nized that additional program-level indicators were 
needed [28]. 

Most recently, PEPFAR changed its monitoring and 
evaluation framework, calling for increased host country 
ownership, greater alignment of indicators with host 
country reporting needs, and inclusion of indicators 
measuring coverage and quality of services [31]. In 
addition, since 2006, PEPFAR has implemented a QI ini- 
tiative, HEALTHQUAL, which helps participants mea- 
sure key indicators and develop QI programs to meet 
improvement goals [26]. HEALTHQUAL programs are 
currently utilized in several PEPFAR countries and many 
of the indicators endorsed are program-level measures 
defined to meet program and country needs [26,32] . 

The continuum of HIV care as a framework for 
developing quality improvement measures 

Despite improved care resulting from scale-up of the 
global HIV/ AIDS response, substantial morbidity and 
mortality can still be traced to inadequacies in care deli- 
very at distinct points along the HIV care continuum 
(Figure 1). These critical points include diagnosis, linkage 
to care, initiation of antiretroviral therapy, treatment or 
prophylaxis of opportunistic infections, and retention in 
care over time [8,33]. Successful patient outcomes are 
contingent upon a high degree of success at each and 
every point on this continuum [34,35]. While barriers to 
optimal care are not yet fully elucidated, quality indica- 
tors focused on essential points along the care con- 
tinuum will be necessary to identify weaknesses and 
design interventions to maximize performance. In the 
descriptions below we highlight the seven key process 
steps in the HIV care continuum; to: 1) summarize the 
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Table 1 Summary of current HIV testing, treatment, and care indicators 



STEP ALONG CONTINUUM 



INDICATORS 



UNAIDS 



Global Fund for HIV, TB, & Malaria 



PEPFAR 



a) HIV Testing & Diagnosis 



b) Linkage to Care 

c) ART Eligibility: Clinical, 
Laboratory, & Psychosocial 
Assessment! 



d) ART Preparation: Literacy 
Training, 01 Prophylaxis, and 
Adherence Assessment! 

e) ART Initiation 



f) Retention in Care 



Clinical Outcomes 



Other 



% high risk persons HIV- 
tested with known result last 
12 months 

% M & F 15-49y HIV tested 
with known result last 12 
months 



% patients with incident TB 
treated for TB & HIV 



% adults & children with 
advanced HIV receiving ART 



% adults & children known to 
be on treatment 1 2 months 
after ART initiation 



% high risk persons HIV-tested with 
known result last 12 months 



% M & F 15-49y HIV-tested with known 
result last 12 months 



% sexually active M & F 1 5-24y HIV-tested 
with known result last 12 months 



% eligible adults & children that received 
co-trimoxazole prophylaxis 



% adults & children with advanced HIV 
receiving ART 

% ART facilities monitoring CD4 in 
accordance with national guidelines 

# and% persons starting ART who picked 
up all ART drugs on time 

% ART facilities that have experienced a 
stock-out of at least one required ART 
drug in the last 12 months 



i health facilities that offer ART 



# persons HIV-tested & received 
result 



% M & F 15-49y HIV-tested with 
known result last 12 months 



% patients screened for TB 

% patients that started TB 
treatment 

# malnourished patients that 
received therapeutic or 
supplementary food 

# eligible clients that received 
food &/or nutrition services 

# patients that received co- 
trimoxazole prophylaxis 

% adults & children with advanced 
HIV receiving ART 

# adults and children with HIV 
infection newly enrolled on ART 

# adults and children with 
advanced HIV infection receiving 
ART 



% of adults & children known to 
be alive & on treatment 12 
months after ART initiation 



# service outlets providing ART 
services according to national or 
international standards 

# health workers trained to deliver 
ART services according to 
published standards 



A-F are points on the HIV care continuum shown in Figure 1; ART: antiretroviral therapy, TB: tuberculosis: Global Fund recommended indicators, disaggregated by 
sex, age, and pregnancy status: Ireference group is HIV positive patients enrolled in HIV care. 



evidence regarding performance at each step, and 2) 
present our assessment of the opportunities for perfor- 
mance measure development. 

a. HIV testing and diagnosis 

The entry point to HIV treatment is through testing and 
diagnosis, yet only 30% of women and 17% of men in 
Sub-Saharan Africa are aware of their HIV status [4]. 
Historically, HIV testing has been provider-driven and 



prompted by symptoms manifested in the advanced 
stages of HIV disease [36]. Further, uptake of voluntary 
counseling and testing (VCT) at health service sites has 
been estimated at under 50% in several Sub-Saharan 
African countries [37,38]. Recognizing the importance of 
expanding HIV testing, the WHO recommended pro- 
vider-initiated, opt-out HIV testing in 2007 [39]. In 
addition to health service-based VCT, other models, 
including free-standing, mobile, community-based, 
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HIV Testing 
& Diagnosis 




ART Preparation 
Literacy Training, 01 
Prophylaxis, Adherence 
Assessment 




Pre-enrollment LTFU, Ols or Death 



Pre-ART LTFU, Ols, or Death 



Post-ART LTFU, Ols, or Death 




Figure 1 The Continuum of HIV Care in Resource Limited Settings. Points A-F represent distinct structures, processes, and outcomes along 
the HIV care continuum. Triangles (A) represent structures, ovals Of) represent processes, and rectangles (=) represent outcomes. ART: 
antiretroviral therapy, LTFU: lost to follow-up, VCT: Voluntary Counseling and Testing, Ol: opportunistic infection. 



home-based, and self-testing have also been studied 
[36,37,39-42]. Routine monitoring of VCT uptake may 
inform the most successful models. 

Quality measures for HIV/AIDS treatment programs 
should first assess how the system performs with regard 
to HIV testing. The current UNAIDS, PEPFAR, and 
Global Fund reports contain indicators that measure 
HIV testing in several sub-populations [1]. Given the 
poor uptake of most VCT models to date, additional 
indicators highlighting the adoption of alternative VCT 
models (such as routine, healthcare-based, or home- 
based) could be used to provide benchmarks for future 
performance [36]. 

b. linkage to care 

After diagnosis, HIV-infected patients must be success- 
fully linked to treatment programs. There is a paucity of 
data on linkage to care in resource-limited countries. 
Studies in southern Africa suggest that 50-70% of 
patients who tested positive for HIV enrolled in clinical 
care within 3-9 months of diagnosis, representing the 
largest lost opportunity to engage patients along the 
HIV care continuum [6,7,43]. In one systematic review 
of HIV treatment programs in Sub-Saharan Africa a 
median of 59% of patients successfully link from HIV 
testing to CD4 testing or clinical staging [44]. Features 
such as multiple care sites (e.g. for HIV testing, CD4 
count testing, and tuberculosis treatment), long wait 
times for appointments and receipt of test results, in 
addition to transportation barriers and medication costs 
are obstacles to effective linkage [8,10,45,46]. Definitions 
of linkage to care are not straightforward [44]. In some 
clinical settings, patients may report to a facility outside 
of a clinic for CD4 testing, and thus may obtain CD4 



testing without being linked to care in an HIV treatment 
program. In addition, some have argued that patients 
who are not yet eligible for ART at the time of HIV diag- 
nosis may need to have multiple clinical visits to demon- 
strate effective linkage to care [44,46]. 

While the barriers to linkage to care are not fully 
understood, indicators that assess delays in these steps, 
such as time from diagnosis to enrollment in care can 
still be defined [6,7]. Such measures are currently lack- 
ing, and are challenging to programs that do not have 
health system infrastructure that supports patient identi- 
fication across different healthcare providers. Nonethe- 
less, measures for linkage to care must identify patients 
who have enrolled in HIV clinics. A recent systematic 
review of retention in HIV care from testing to treat- 
ment in Sub-Saharan Africa calls for standardization of 
terms to facilitate the study of linkage to care across 
multiple care sites. The authors define 2 stages addres- 
sing care linkage. Stage 1 begins after receipt of a posi- 
tive HIV test, and ends with the receipt of CD4 count or 
clinical staging result and referral to ART or pre-ART 
care; stage 2 begins after referral to pre-ART care, and 
ends with ART eligibility [44]. Adoption of such stan- 
dard definitions would be an important step in develo- 
ping quality measures. 

c. Antiretroviral therapy eligibility 

Once enrolled in HIV care, patients should be staged 
with clinical and psychosocial evaluation in addition to 
CD4 count testing to assess eligibility for antiretroviral 
therapy, and to screen and prophylax for opportunistic 
infections [47]. Given evidence that starting treatment 
earlier reduces the risk of AIDS and death, the WHO in 
2009 increased the recommended threshold for ART 
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initiation from <200 cells/uL to <350 cells/uL [48]. 
Nonetheless, patients in most low-income countries 
present to care much later (median CD4 108 cells/uL) 
than their counterparts in high-income countries 
(median 234 cells/uL) [11]. This late presentation is 
clearly associated with an increased risk of death [11,48]. 
Further, recent evidence from HPTN 052, a trial of anti- 
retroviral therapy in couples with discordant HIV status, 
showed significantly less transmission when antiretro- 
viral therapy was initiated with a CD4 cell count of 350- 
500 cells/uL compared to less than 250 cells/uL. Thus, it 
is likely that antiretroviral therapy eligibility will move 
even earlier in disease, and will present new challenges 
to LMIC [49]. 

Tuberculosis (TB) is the leading cause of illness and 
death among people living with HIV worldwide [50]. 
Screening for TB upon entry into HIV clinic, prompt 
treatment for active TB, and disease prophylaxis were 
recommended by the WHO as priority interventions in 
2010, but uptake is poor [50]. In 2009, only 5% of the 33 
million people living with HIV were screened for TB, 
and optimal screening methods are yet to be established 
[51-54]. 

Though not standard practice, increasing program- 
matic support has also focused on food supplementation, 
given the importance of food insecurity and malnutrition 
on outcomes of HIV-infected patients [55-57]. Recent 
data from resource-limited settings demonstrates that 
food assistance is associated with increased body mass 
index and better clinic adherence [58,59]. In addition, 
one study from rural Uganda suggests that early initi- 
ation of antiretroviral therapy may improve physical 
health status and thereby improve food security [60]. 

d. Antiretroviral therapy - preparation and initiation 

After disease staging, most HIV treatment programs 
have incorporated a process to assess antiretroviral ther- 
apy readiness that includes psychosocial assessment, 
adherence counseling, and identification of a treatment 
supporter. This process typically spans several clinical 
visits [61-63]. Many programs describe long delays in 
antiretroviral therapy initiation, with patients waiting up 
to 120 days to complete adherence training [8]. Stock- 
outs of essential drugs affect at least 11% of patients on 
antiretroviral therapy treatment, according to one study 
in Cote d'lvoire. Resultant treatment discontinuation is 
associated with increased risk of care interruption or 
death [64]. Though poor adherence to antiretroviral 
therapy is associated with increased mortality, the opti- 
mal format and duration of antiretroviral therapy literacy 
training, and its impact on adherence, have not been 
established [65]. One study in Uganda found that com- 
pletion of adherence training prior to ART initiation did 
not improve adherence [66]. In addition to these delays, 



16-45% of patients are also lost from care before starting 
antiretroviral therapy, and others die while waiting to 
start treatment [8,44,46,67]. 

Substantial delays also exist in the time from enroll- 
ment in care to full clinical assessment and ART initi- 
ation [6,7,61]. Potential indicators could measure the 
percentage of patients enrolled in an HIV care program 
that receive a CD4 count within 3 months of HIV diag- 
nosis, or the delay to initiation of ART in eligible 
patients. Indicators could also be defined to determine 
the proportion of patients screened for concomitant TB 
infection, which is a critical problem in those who are 
HIV-infected. 

e. Retention in care 

For patients enrolled in antiretroviral therapy programs, 
high program attrition rates have been described, with 
15-55% of patients lost from care between 6 and 
36 months [9,10,45]. In programs that have actively 
traced patients lost to follow-up, up to 40% were found 
to be dead [8,10,63]. Some patients transfer care to other 
programs, and others cycle in and out of care with brief 
interruptions but worse virologic control than those who 
remain in care [68,69]. As a result, patient retention has 
been increasingly recognized as an important program 
indicator. 

Once initiated on antiretroviral therapy, patients 
should be monitored for medication adherence and 
toxicity in accordance with local guidelines. These efforts 
require reliable medication supply chains and laboratory 
services, which can be assessed through specific indica- 
tors such as the number of stock-outs of essential medi- 
cations per 6-month period. These measures would 
identify the greatest delays in the system, potential inter- 
ventions, and appropriate targets for future performance. 

f. Clinical outcomes 

Optimally, HIV-infected patients will remain in care, on 
antiretroviral therapy, with controlled disease. However, 
patients in resource-limited settings who are initiated on 
antiretroviral therapy are 3.5 times more likely to die 
than patients in resource-rich countries [11]. This risk of 
death is highest in the first few months after antiretro- 
viral therapy initiation, and has been attributed to pres- 
entation to care with advanced disease [12,70]. Despite 
the increased risk of early death, patients in resource- 
limited and resource-rich settings appear to gain similar 
immunologic and virologic benefit from antiretroviral 
therapy [71-73]. Approximately 70% of those on therapy 
achieve virologic suppression at 6 months [11,72]. WHO 
has provided guidelines for treatment failure based on 
CD4 count response, but discordant virologic and im- 
munologic responses to antiretroviral therapy may occur 
in up to 20-30% of patients, and viral load monitoring is 
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not widely available in many resource-limited settings 
[47,74,75]. Patients who do fail therapy may require in- 
tensive adherence interventions, or switching to more 
costly second and third-line treatment regimens [76] . 

Improved data collection mechanisms, unique patient 
identifiers, electronic medical records, use of patient 
trackers, and program cost subsidies may reduce loss to 
follow-up [10,14,45]. UNAIDS and PEPFAR define one 
indicator that measures 12-month retention rates, but 
the complementary measure, loss to follow-up is less 
well defined [77]. Geng et al. suggest a sample-based 
approach to estimate the outcomes of these patients, 
which may be appropriate to help programs understand 
barriers to retention [78]. In addition to patients 
initiated on antiretroviral therapy, it is important to 
monitor patients who are antiretroviral therapy ineligible 
at the time of enrollment, to ensure that they are not 
lost to follow-up despite earlier presentation to care. An 
indicator measuring the percentage of patients antiretro- 
viral therapy ineligible at baseline who receive a follow- 
up CD4 count in 6 months could capture this. 

An indicator scorecard for HIV programs in resource- 
limited settings should ultimately contain metrics 
reflecting clinical and patient-reported outcomes. These 
should be important measures in their own right, or 
have known association with clinical outcomes, such as 
retention in care, mortality (both before and after anti- 
retroviral therapy initiation), as well as treatment failure 
and switches to second-line therapy. 

g. Patient-reported outcomes: health-related quality of life 
and patient satisfaction 

The Institute of Medicine defines patient-centered care 
(care that is respectful and responsive to individuals) as 
one of the six aims for improving the delivery of quality 
health care [79]. Patient- reported outcomes such as 
quality of life and service satisfaction are examples of 
measures that characterize patient-centered care [80]. 
Health related quality of life (HR-QOL) has been 
increasingly recognized as an important outcome, par- 
ticularly as HIV has transformed into a chronic disease 
in the effective ART era [81,82]. HR-QOL is a complex 
measure that includes several dimensions including 
physical function, symptoms, performance of social 
roles, emotional status, cognitive functioning, and indi- 
vidual feelings about health [80,83,84]. Several HR-QOL 
scales have been developed for use across international 
settings (WHO-QOL, PRO-QOL) and for HIV research 
in research-rich settings (MOS-HIV, FAHI, HOPES, 
HAT QOL, AIDS-HAQ, MQOL-HIV) [83-90]. Most of 
these instruments were developed in the pre-ART or 
pre-effective ART era, are long, and were developed for 
research rather than routine clinical practice. There is 



no consensus on the optimal tool for measuring HR- 
QOL among patients infected with HIV [81]. 

Several studies have described a range of challenges for 
poor patients seeking medical care in resource-limited 
settings and across the globe, such as difficulty accessing 
care, high direct and indirect costs of care, long wait 
times, and poor treatment by staff [10,45,78,91-94]. 
These challenges can all influence patient satisfaction, 
defined as the extent to which a patient's health care 
experience matches his or her expectations [95]. There 
has been relatively little focus on patient satisfaction with 
HIV care services in resource-limited settings [96-101]. 

More effort is needed to identify and standardize 
metrics for quality of life and care satisfaction that can 
be used in routine clinical practice in resource-limited 
settings [84]. The AIDS Clinical Trials Group (ACTG) 
has used a brief 2-question measure (one 5-point likert 
scale, and another rating on a scale of 1-100) to evaluate 
patient reported health status in the ACTG Longitudinal 
Linked Randomized Trials protocol [102]. Health status 
measures a dimension of quality of life that is impacted 
by health care satisfaction [83-85]. These ACTG mea- 
sures may provide a useful starting point for the 
assessment of important patient reported-outcomes to 
promote patient-centered care. Because of the con- 
straints posed by limited resources, it becomes even 
more critical to focus on indicators that will have the 
greatest impact on each health system and the patients it 
serves [14]. 

Proposed quality measures for HIV programs in resource- 
limited countries 

The set of quality indicators currently used to assess the 
treatment and care of HIV programs in resource-limited 
countries lacks focus on several areas most likely to 
improve HIV outcomes (Table 1) [11,77,103]. The ten- 
sion between funder needs for accountability and stra- 
tegic planning, and evaluation and program needs for 
local improvement, may be difficult to harmonize with a 
single set of indicators. Therefore, using the model in 
Figure 1 as a framework, we assessed the literature 
describing the operational challenges facing HIV care in 
resource-poor settings. Utilization of this framework, 
along with the selection of indicators measuring both 
processes of care and key clinical and patient reported 
outcomes guided the development of a set of 24 
program-level measure concepts to guide improvement 
for HIV treatment programs (Table 2). The measure 
concepts were designed to address the limitations of 
current indicators described above, as well as Utilization 
of this framework, along with Utilization of this frame- 
work, along with the NQF criteria of importance, usabi- 
lity, and feasibility. 
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Table 2 Proposed scorecard of program-level quality measures 


STEP ALONG CONTINUUM 


MEASURE 


a) HIV Testing and Diagnosis 


■ % of patients diagnosed on site [P] 




■ % of patients diagnosed in other medica facilities [P] 




■% of patients diagnosed via home-based test [P] 




■ % of patients diagnosed in mobile testing unit [P] 




■ % of couples whose partners have been HIV tested and are aware of results [P] 


b) Linkage to Care 


■ Median days from HIV diagnosis to referral for ART or pre-ART care [P] 




■ % of patients ART ineligible at baseline who receive a follow-up CD4 count in 




6 months [P] 




■ Median days from clinic enrollment to ART eligibility [P] 




■ % of patients who are enrolled in HIV clinic, received CD4 count & results within 




3 months of HIV diagnosis [P] 




■ % of patients with CD4 count < 200 ce s/uL, & < 350 cells/uL at presentation [P] 


c) ART Eligibility: Clinical, Laboratory & Psychosocial 


■% of patients screened for tuberculosis [P] 


Assessment 


■ % of eligible patients provided with nutritional supplementation [P] 




■% of ART-eligible patients who died before ART initiation [O] 


fi\ ART Prpnaratinn* I itprarv Traininn fll Pi"nnh\/lavic 
\a / nn i ric|Ja[c]iiuii< i_i lci cii_y i i on iii ly , \ji n \J \J 1 1 y iciaij, 


■ K/lpnipn na\/<: from pnrn mpnt to ART initiation for p inih p o:atipntc: TP1 

■ ivicuidii udyj iiuiii ci 11 1 iCi 1L lu r\v\ I n iiuaLiui I iui cnyiLJic uaLit:i I Lza [nj 


and Adherence Assessment 


■ Median days from enrollment to completion of ART literacy training for eligible 




natipnt 1 ; TP1 


e) ART Initiation 


■ # drug stock outs in last quarter for first-line ART drugs or cotrimoxazole [P] 




i I'll" i r ^r>>ij-j-" „j I "4_- 4-jr 

■ Local guideline concordance (e.g. CD4 testing, adherence monitoring, assessment for 




rinin tnviritv Ol c;rrppninn Rj nronh\/lavi^l TP1 

\J 1 IUA 1 V . 1 Ly f \y\ j^lTZTd III ILJ 13! LJIWLJI 1 V CIAI J J L^J 


f) Retention in Care 


■ % of patients retained in care 6 and 1 2 months from enrollment (ART eligible and ART- 




ineligible) [O] 




■% of patients who transferred care to other clinics 6 and 12 months from enrollment 




[0] 




■% of patients deemed lost to follow-up who have been contacted by clinic staff to 




determine outcome [0] 


g) Clinical Outcomes 


■ % of patients on ART with undetectable viral load at 12 months [O] 




■ % of patients on ART requiring switch to second-line therapy for treatment failure at 12 




and 24 months [O] 




■ % of patients (ART-eligible on and off ART) who died 12 months after enrollment [0] 


h) Patient Reported Outcomes 


■ patient-reported health status 6 and 12 months after clinic enrollment [0] 



[P] = process measure, [O] = outcome measure. 



To assess HIV testing and diagnosis, we propose five 
process measures to illustrate where patients who suc- 
cessfully present to clinic receive their testing (clinic- 
based vs. home or mobile testing), and to inform whether 
partners of known HIV-infected patients are receiving 
testing. These data will allow HIV treatment programs to 
provide critical feedback about which local HIV testing 
modalities are most successful and may benefit from 
scale-up, in addition to which are least successful and 
could benefit from improvement efforts. 

To determine program performance around linkage to 
HIV care, we recommend five process measures measur- 
ing both stage 1 (from receipt of a positive HIV test to 
receipt of CD4 count or clinical staging result and refer- 
ral to ART or pre-ART care) and stage 2 (from referral to 



pre-ART care to ART eligibility) linkage as defined by 
Rosen et al., [44]. The most effective measurements of 
linkage to care require the merging of data from HIV 
testing and treatment centers, which are rarely available 
in resource-limited settings [44,46]. Our proposed mea- 
sures can be captured at the point of HIV care alone. 

We propose six process and one outcome measure to 
assess program performance on it's efficiency in deter- 
mining ART eligibility and effectively initiating patients 
on ART. These include screening for opportunistic infec- 
tions and nutritional needs in addition to ensuring 
adequate drug supply, measuring the time it takes to 
identify and initiate ART-eligible patients on life-saving 
therapy, and concordance with local ART monitoring 
guidelines. The proposed outcome measure provides 
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programs with important data on potentially preventable 
deaths of ART-eligible patients who die before starting 
therapy. 

Finally, we outline five outcome and one process meas- 
ure addressing retention in care, patient health status, 
and important clinical outcomes (virologic suppression, 
use of second-line therapy, and mortality). Our measures 
of care retention, in addition to assessing this as an 
outcome, also necessitate that HIV care delivery pro- 
grams ask whether they have intervened to decrease 
LTFU and improve retention. This set of measures 
encompasses data on endpoints important to patients, 
clinicians, and administrators. 

Discussion 

Our proposed measurement framework and set of pro- 
gram-level quality indicators have several strengths. The 
HIV care continuum is an increasingly utilized paradigm 
for implementation practice and research in HIV service 
delivery, and thus anchors quality measurement around 
a familiar construct for service providers. Building our 
framework around the HIV care continuum also allows 
us to take advantage of implementation research identi- 
fying weaknesses in the care continuum, and potential 
areas for quality improvement. Our identification of 
measures reflecting processes and outcomes of care 
moves away from volumetric endpoints that highlight 
quantity rather than quality of services. Finally, 
standardization of a set a quality measures allows for 
comparison of performance across programs, enabling 
the formulation of benchmarks for appropriate care and 
identifying best practices in care delivery. 

Taken in combination, the proposed measures form 
the basis of a balanced scorecard of crucial, evidence- 
based objectives for successful HIV service delivery pro- 
grams in resource-limited settings. The proposed meas- 
ure concepts will require further specification and 
standardization before they can be implemented as indi- 
cators (as described above). As is the case in the devel- 
opment of performance measures, some will prove more 
useful than others; this is difficult to predict prior to 
implementation. Because program strengths and weak- 
nesses differ from site to site, some programs will find 
certain measures more useful than others. 

These proposed measures also have limitations. The 
NQF suggests that quality measures meet 4 criteria: im- 
portance, usability, feasibility, and scientific acceptability. 
While we considered the NQF's criteria for performance 
measures (importance, usability, and feasibility), in ma- 
king recommendations, the latter two components can 
only be fully appreciated once measures are specified 
and field-tested. Usability and feasibility will invariably 
depend upon the data collection systems available at the 
program level. An assessment of scientific acceptability 



(reliability and validity) of the proposed measures will 
also be important. Finally, measurement alone will not 
improve the quality of patient care without being linked 
explicitly to the dissemination of data to care teams, and 
empowering these teams to adapt care processes to 
improve performance and the quality of care provided. 
Building this type of approach to quality improvement 
into organizations in well-resourced countries has proved 
challenging, so there will undoubtedly be challenges in 
resource-limited settings as well. 

Conclusions 

An aphorism in quality improvement science ascribed to 
one of its founders, William Deming, states; "Every sys- 
tem is perfectly designed to achieve exactly the results it 
gets." With the rapid scale-up of life-saving therapy for 
HIV/ AIDS, it is important to consider the quality 
challenges in the system that has emerged for this care 
in resource-limited settings. The challenges are high- 
lighted by inadequate HIV testing and late presentation 
to care, poor linkage, major delays in ART initiation, 
and high loss to follow-up rates, all of which limit the 
life-saving gains that the system aims to promote. To 
improve the quality of care provided, identification and 
measurement of a quality indicator scorecard that 
reflects important processes and outcomes along the 
continuum of HIV care is critical. There is now substan- 
tial global commitment to improving the quality of HIV 
care in resource-limited settings. Focused efforts to 
define and improve upon the most important quality 
indicators for HIV-infected individuals are crucial as 
programs continue to grow. 
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